Two Methods for the Practical Analysis of Rating Data

Managing human judgements is complicated by the fact that ratings rarely agree perfectly. This paper presents two methods to handle such disagreement. The rst method is to remove inconsistent ratings and it will be shown that this enhances the quality of the data. Further assumptions about the relation between ability and assessability of subjects allows us to recover the information in the inconsistent ratings using a generalized partial credit model (GPCM). The use of the GPCM is illustrated with real data.